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Abstract 


Consider  an  observed  stochastic  process  consisting  of  a  signal  with 
additive  noise.  Asstime  that  the  signal  has  finite  energy  and  that  the  signal 
and  noise  are  Independent.  In  this  paper  we  show,  that  under  the  above  assump 
tlons  the  Innovations  and  observations  a-algebra  are  equal  thereby  proving  a 
long-standing  conjecture  of  Kallath. 
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Introduction. 


(J2,F,P)  te  a  complete  probability  space,  F  =  (F^)»  0£t<l,  a 
non-decreasing  family  of  sub^ -algebras  and  W=  (W^,F^),  0  £  t  £  1,  a  Wiener 
process.  With  a  signal  process,  8  = 

as  observations,  the  innovations  problem  is  to  determine  whether  y  =  (y^,F^)  is 
adapted  to  the  innovations  process,  (v,F^).  This  process,  whenever  it  exists 
(see,  for  example,  [l]),  is  a  Wiener  process  defined  by  the  equation 


(2)  V, 


where  S  =  E(8  |y  ,  0  <  s  £  t).  The  innovations  problem,  first  posed  by  Kailath 
in  1961  and  subsequently  considered  by  Frost  in  his  thesis  1 2]  can  be  posed  in 
probabilistic  terms;  namely,  are  the  o-algebras  generated  by  these  processes  the 
same  modulo  null  sets;  i.e.  is 


a{y  :  s  <  t}  =  a{v  ,  s  <  t}  (mod  P)? 
"s  —  s  — 


In  this  paper,  we  show  that  in  the  form  conjectured  by  Kailath  [3]  this  problem 
has  a  positive  solution.  Our  assumptions  are  that 

(a)  Signal  and  noise  processes  are  independent 

and  P 

(b)  E(  /  B  ds)  <  «  . 

•'0  ® 

Our  results  generalise  all  known  results  on  the  innovations  problem 
([^Is  [3]).  In  [^]  the  signal  process  is  assumed  to  be  uniformly  bounded.  The 
proof  given  in  [2]  is  incorrect  (see  [3]).  This  problem  has  also  been  considered 
by  Benes  [5]  and  Kallianpur  [6]  under  slightly  weaker  hypotheses  than  ours.  Their 
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proofs  however  appeea*  to  he  incorrect.  We  have  been  Informed  that  results 
similar  to  ovirs  have  been  independently  obtained  by  J.M.C.  Clark  and  M,P.  Ershov. 

The  problem  considered  here  is  a  subclass  of  the  more  general  innovations 
problem  for  stochastic  differential  equations  ([?  ]  Page  26o).  In  this  more 
general  form,  the  innovations  problem  does  not  have  (in  general)  a  positive  solu¬ 
tion.  A  counter-example  was  given  by  Cirelson  ([  7],  Page  150 ).  In  Cirelson's 
example  no  "filtering"  takes  place  and  thus  it  cannot  be  considered  to  be  a 
counter-example  to  the  innovations  problem  for  non-linear  filtering.  Cirelson's 
example  however  can  be  modified  to  obtain  examples  where  filtering  does  occur 
(cf.  Benes  [8]).  The  proof  presented  in  this  paper  utilizes  the  independence 
of  the  signal  and  noise  processes  in  an  essential  way.  Nevertheless  we  feel  that 

the  assvuaption  of  independence  can  be  removed  for  a  wide  class  of  signal  processes. 
1.  The  Innovations  Result  under  (a),  (b). 

Our  proof  consists  of  two  parts:  deriving  a  jointly  measurable  functional, 
Y(s,x), 

Y:  [0,1]  X  C[0,1]  R 

with  the  property  that 

y(s,  6((»>))  =  8  (u)  X  X  p  _  a.s. 
s 

(X  denotes  Lebesgue  measvire  on  [0,l]),  and  then  showing  that  any  weak  solution 
of  the  stochastic  differential  equation 

(U)  =  Y(t,C)dt  +  dv^ 

is  pathwise  unique  in  the  sense  of  Yamada  and  Watanabe  [ 9  ] .  It  is  a  consequence 
of  their  work  that  (U)  has  a  strong  solution  (in  the  sense  of  Ito)  i.e.  the  obser¬ 
vations  are  a  functional  of  the  innovations. 

Under  (a),  (b),  we  may  apply  the  results  of  Kallianpur  and  Striebel  [lO]  to 
show  that  for  0  <  t  <  1, 


-3- 


(5)  3^(w) 


/fi  /o  Jo  Pg(“)ds)dP{u) 

J exp(J^  &^(oi)dy^((a)-isJ^  Bg{u)d8)dP(a)) 


—  0)  a.  8. 


Replacing  y  by  x  for  x  e  C  [0,1],  we  surrive  at  Y{t,x).  The  Joint  measurability 
s  s 

of  Y(t,x)  rests  upon  the  measurability  in  {t,ti),x)  of  the  functional,  (or 
stochastic  integral). 


<  D3  (u),x 


which  represents  a  Gaussian  random  variable  with  respect  to  Wiener  measure 
on  C[0,t]  whenever  3{(o)  is  in  L2[0,t],  for  t  ^  1.  The  operator,  D,  is  unitary 
from  LgtO,!]  onto  C,  the  Hilbert  Space  of  continuous  function  with  square  in- 
tegrable  derivatives,  and 

Df(8)  = 

(Further  discussion  is  given  by  Kuo  [ll],) 

Our  hypotheses  (a),  (b)  guarcuitee  that  the  innovations can  be  constructed 
[l]  and  so,  (U)  is  satisfied  by  the  observations.  To  show  that  emy  weak  solution 
to  (4)  is  pathwlse  unique,  we  will  need  the  following  lemmas. 

Lemma  1.  Let 

p(t,x,u)  “expC^Q  6g(u)dXg  -  hJq  ^g(u)dB) 

and 


Then 

(a)  VU,  (x:  sup  g(t,x)  <  »}  =  1 

"  0<t<l 


(b)  P-,  {x:  inf  g(t,x)  >  O)  =  1  . 

vhere  is  Wiener  measure  on  C[0,l]. 

Proof:  Recall  that 

^  ^1  ^  ^ 

P{a):  /-  3^  a){ui)ds  <  “}  =  1, 

JO  s 

A 

and  for  each  such  u,  the  process 
{p(t,W((jj),ijj),  F^} 

w 

is  a  (right)  continuous  martingale.  Consequently,  {g(t ,W(a)) ) ,F^}  is  a  right 
continuous  martingale.  For  let  X(W)  he  a  hounded  F^-measurahle  random  variable, 
8  <  t. 

Then 

Ep[X(u)  .  g(t,W)] 

=  /(,X(x)  g(t,x)  dy^(x) 

=  [  J^X(x)p(t,x,a))  dy^^Cx)]  dP(w) 

*  /n  [/c  dy^(*)]  clP(u)) 

*  /c  g(8,x)  dy^(x), 

from  which  we  obtain 

Ep(g(t,W)lFg)  =  g(8,W)  -  0)  a.s. 

w 

Moreover,  since  the  family  of  sub-a-algebra,  {F  ;  0  s  ^  1/  is  continuous,  and 
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Ep(g(t,W))  =1,  0  <  t  <  1» 

it  follows  that  {g(t,W),  F^}  has  a  right  continuous  version  [Thm.  3.1,  7]. 
Thus,  we  conclude  that 


P{w:  sup  g(t,W(a)))  <  oo}  =  1 

q<t<i 

since  PCw:  sup  g(t,W(ai))  >  X}  £  1/X 

0<t<l 

for  X  >  0.  This  gives  (a). 


dy.. 


For  (h),  note  that  g(t,x)  is  the  Radon-Nikodym  derivative,  (t,x) 

where  y  Is  the  measure  induced  on  C[0,l]  hy  the  ohservations  process,  y. 
y 

Since  the  proof  is  analogous  to  Lemma  6.5  t7  ]. 


Lemma  2.  Let 


a(t,x)  »  L  [  f  3^(2))ds]*p(t,x,w)  dP(u). 

-'n  -'o  ® 


Then 


y^  {x:  sup  a(t,x)  <  »}  =  1 
^  0<t<l 


Proof;  The  process 

{a(t,W(oj)),  is  a  right  continuous  martingale_ and  for  0<t<l, 
Ep(a(t,W))  =  /n  (/o  3g(w)ds)  dP(u). 


Lemma  3.  Let 


m(t,x)  »  dP(u)). 

Then 

y^  {x:  m(t,x)dt  <  «}  =  1  . 
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Proof :  Observe  that 

r  if  ^  in{t,W{a)))dt]  dP(a))  *  f  ^  L  dP(w))dt  <  ». 

•'fl  •'0  ■'o  ■'n 

We  ret\irn  to  the  problem  of  comparing  two  weak  solutions  5^,  of  (*+)» 

assuming  that  Cq»  are  both  defined  on  the  space  (fi,F,P).  Moreover,  we  may 

assume  that  »  Pr  are  each  absolutely  continuous  with  respect  to  Wiener  measure. 
^0  ^1 

The  proof  of  Proposition  1  [  9  ]  remains  valid  with  respect  to  the  restricted  class 

of  solutions  C  which  satisfy  the  condition 

P  [  f  ^  (Y(t,C))^dt  <  «]  =  1. 

•'0 


See,  in  particular,  pg.  l6l  [  9  !•  Prom  Lemmas  1,3, 
aure  equivalent  to  Wiener  measure  since 


it  follows  that  Vc  » 
^0 


y^  {x:  J\y{t,x)f  dt  <  ~>  «  1  . 

Thus,  for  i  =  0,1,  we  conclude  that 


sup  g(t,  ((*>))  <  “ 
0<t£l 


inf  g(t,  >  0 

0<t<l 


sup  a(t,  £;j(w))  <  « 

0<t<l 

,1 

and  /  m(t,  (a))  ).dt  <  <»  -  o)  a.s. 

•'0  ^ 
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Theorem  1.  If  are  weak  solutions  of  (4),  then 


sup  I  C  (t,w)  -  CT(t,a))|  =0  P  -  a.s. 

0<t<l  ^ 


Proof.  On  [0,1]  C[0,l],  define 

iCt.co)  =  |Y(t,CQ)-  Y(t,C^)|  =  I  -  I 

f(t,C  )  -  f(t,C,)  /  ,  1  \  1 

g(t,SQ)  *  l^gltTI^  "  gTtTf^j  I 

Then 

(6)  (L(t,a>)f  <  K(a))[(f{t,CQ)  -  f(t,C^))^  +  {f(t.q) )^[g(t,Co)  -  eit,^^)f) 


where  K(oj)  >  max 


( 


inf(g(t,5Q))‘ 


0<t<l 


inf(g(t,CQ))' 

0<t<l 


inf(g(t 

0<t<l 


For  0  <  u  <  1,  we  write 


(7) 


f  L^(t,(a)dt  <  K(a))|  J  |J  |3^(w)  |  |p{t,?Q,a))-p(t,?^,a))  |dP{w)  |  ^dt 

+  (f(t,C^))^-|  |p(t,CQ,u}.p(t,c^,£)|dp(w)|^dt 


P 
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and  because  e*  is  convex,  it  follows  for  all  u,  t,  that 
(8)  |p(t,CQ,a))  -  p(t,C^,a))| 

1  ^  lp(t,CQ,u))  +  p(t,Cj^,u))l  •  1  8^{u)a(£;— C^(a)))l 

=  |p(t,CQ,a))  +  p(t,Cj^,a))  I*  I  8^(w)L(s,ai)ds| 


Applying  Holder's  inequality  to  the  last  integral  term  in  (8),  and  bringing 

out  (  r  ^(L(s,w))^ds)^^^,  yields 
■'O 


(9)  (i(t,a))  )^dt  ^  K(a))  (i(s,w))^ds]  •  t/>(t,to)dt 


-  u)  a.s. 


where 


'/'(t.oj)  =  [  |8^(a))||  (p(t,CQ,w)+p(t,q,a))|.(y  8g(a))ds)’^^  dp]^ 


+  [J  ^6^(a))p(t,C^,a))d.P3 


P(t.Co,i)*P(t.C^.S)  (  I*  ap(£)l^  . 


By  showing  that  <|^(t,(D)  is  an  integrable  function  of  t  -wa.s.,  one  may  then 
iterate  (9)  and  conclude  that  for  0  <  u  <  1, 


2 

J  (L(t,a)))  dt  =  0  0)  -  a.s. 

0 


Hence  sup  (t,a))  -  C  (t,w)| 

0<t<l 
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amd  we  have  established  path-wise  uniqueness  for  weak  solutions  of  (U), 

To  see  that  t|/(t,<i))  is  integrahle  in  t,  to-  a.s,  apply  Holder's  inequality 
to  the  first  term  to  obtain, 

(10)  |3^(^)||  p(t.gQ,(o)+p(t,gj^,a))|»^^  $^(a))ds)^^^  dP(ai)  ] 

2  0  ® 

-  ( /^  |e^(oj)|  |p(t,CQ,a))+p(t,C^,(o)|  dP(£o)  )  • 

ji  - 


(a))ds  ]  (p(t,CQ,tI))+p{t,C^,ti)))  dP(a))) 

2 


< 


J  |3^(w)l  {p(t,CQ  <o)+p(t,5j^,io))dP(to)) 

fi  a  ~ 


sup 

0<t<l 


r  ri  5  -s  {p(t,Cn»w)+p(t,C,  .u) 

fj  P  ( - 0„^ - 


)  dP(w) 


Integrating  this  product  over  t  gives 

1  o  ^  (p(t,CQ»u)+P('t.Cl»w) 

(11)  sup  [  f  3  (to)ds]  •  (  2 

o<t_<i  J  n  Jo 


)  dP(u) 


f  r  1  dP(a))  1  dt.  <  "  u  -  a.s. 

0  I  fi  ®  - - 2' 

hy  Lemmas  2,3. 

The  second  term  of  t|j{t,a))  is  handled  analogously  using  Lemmas  1,2,3. 


This  completes  the  proof. 
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Thus,  the  observations  process  {y.},  0<t<l,  is  the  (unique)  strong  solu- 
tion  satisfying  (1|)  under  the  restriction  that 

P(  P  Y(t,?)^dt  <  »)  =  1. 

0 

Final  Remarks. 

Let  us  rewrite  equation  (2)  as 

(12)  V  *  (l-N)y  ,  where  N  is  a  non-linear  operator  from  C[0,l;y  ]  into 

y 

C[0,l;y^].  Under  assumptions  (a)  and  (b)  we  have  shown  that  an  inverse  operator 
(l  +  W)  exists  such  that  P  -  a.s. 

(13)  y  =  (I  +  M)v  . 

Moreover,  if  :  C[0,l]  -►  C[0,l]  denotes  the  truncation  operator  defined  by 

X  ,  0  <  s  <  t 

ilk)  (TT  X)(s)  =  {  ^ 

0  ,  otherwise  , 

then 

^  A 

(15)  (l  +  W)('ff+v)  =  TT  y  ,  that  is,  the  operator  (l  +  W)  is  causal. 

Our  results  in  this  paper  suggest  the  investigation  of  causal-invertibility 
of  non-linear  causal  operators  on  abstract  Wiener  Spaces  (in  the  sense  of  Gross) 
using  methods  of  stochastic  integration  and  martingades .  Such  an  investigation 
would  also  be  of  importance  in  the  theory  of  stochastic  stability  of  feedback 


systems. 


-11- 


Acknovledgaent  s 


It  is  a  pleasure  for  the  second  author  to  thank  Daniel  Ocone  of  the  Mathe¬ 
matics  Department,  M.I.T.  for  first  noticing  that  there  was  a  gap  in  the  proof 
of  the  innovations  conjecttire  in  the  paper  by  Benes;  S.R.S.  Varadhan  for  con¬ 
versations  about  the  Yamada-Watanabe  theorem  and  other  suggestions;  V.  Benes, 
J.M.C.  Clark,  M.P.  Ershov  and  T.  Kailath  for  discussions  on  the  innovations 
problem. 


-12 


REFERENCES 


[1]  M.  Fujlsaki,  6.  Kallianpur  and  H.  Kvinlta,  Stoduut^C  diMeA^nti/lt 

iquatloni,  ioK  th^  nontCma/L  pJiobZejin,  Osaka  J.  Math.  9  (1972),  l9-**0. 

[2]  p.  Frost,  Estimation  in  cowtcuwuA  time,  mntinean.  systems.  Dissertation, 
Stanford  University,  Stanford,  California,  June  I968.. 

[3]  P.A.  Frost  and  T.  Kaiiath,  An  innovations  app/ioacJi  to  teast-squ/wes 
estimation— Pant  III:  norZinean.  estimation  inuihite  gaussian  noise,.  IEEE  Trans, 
on  Automatic  Control  AC-I6  no.  3  (l97l),  217-226. 

[U]  j.M.c.  Clark,  Conditions  ion.  the  one-to-one  eonn.es ponde nee  between  an 
obsenvation  pnoeess  and  its  innovations.  Tech.  Report  no.  l.  Imperial  College, 
London,  England,  1969. 

V 

[5]  v.E.  Benes,  E^ension  oi  CCank's  innovations  equivatenee  theonem  to  the 

case  oi  signal  z  independent  oi  noise,  with  ds  <  “  a.s..  Mathematical 

Programming  Study  5  (1976),  2-7.  ® 

[6]  G.  Kallianpur,  A  tineon.  stocJhastie  system  with  diseontinuous  eontnot. 

Prof,  of  International  Symposium  on  Stochastic  Differential  Equations,  I^oto 
1972,  edited  hy  K,  Ito,  Wiley  Interscience,  1978,  pp.  127-1^0. 

[7]  R.  Lipster  and  A.  Shiryayev,  Statistics  oi  nandom  pnocesses,  J,  genenat 
theony,  Springer-Verlag,  New  York,  1977* 

[8]  V.E.  Benes,  Nonexistence  oi  stnong  nonanticipating  solutions  to  stoch¬ 
astic  VEss  implications  ion  iunctional  VEs,  iiftening  ana  contnol.  Stochastic 
Processes  and  Their  Applications  5  no.  3  (1977),  243-263. 

[9]  T.  Yamada  and  s.  Watanahe,  On  the  uniqueness  oi  Solution  oi  Stochastic 
diiienential  equation,  J.  Math.  I^oto  Univ.  ll  (1971),  155-167. 

[10]  G.  Kallianpur  and  C.  Striehel,  EstimotiJryi  oi  Stochastic  pnocesses: 
anbitnany  system  pnocesses  with  additive  white  noise  ennon,  Ann.  Math.  stat. 

39  (1968),  785-801. 

[11]  H.H.  Kuo,  Gaussian  measunes  in  banach  spaces.  Lecture  Notes  in  Mathematics 
U63,  Springer-Verlag,  Berlin,  1975* 


1 

i 


